Saturday, January 06, 2018

The advantage and disadvantage of using a debugger...

Before you read on, watch this video carefully and see if you notice a pitfall...

Advantage: it helps one locate where something breaks quite quickly...

Disadvantage: it is so fast that it's very easy to skip over the root of why something broke and "just patch things up"...

In the video, the surface problem (where the error manifested) is the concatenation of two strings where an addition of two numbers was expected, but the actual problem (why) is not String + String vs. Number + Number but rather a bit deeper, it is in the way the values are accessed, you call getNumberX(..) and get a string rather than a number, and in this particular case fixing the root problem (i.e. returning a Number) will eliminate a whole class of potential errors while the solution given in the vid will only "mask" the error and lead to code bloat that is very hard to clear up later.

To reiterate, the debugger helps one locate the breakage, and maybe patch a particular case (wrong operation type in the vid) but it does not replace the skill to find/see the actual root of the breakage, the actual bug (here "getNumber returns String" name/return mismatch) and fix it.

Wednesday, June 03, 2009

Strike three!

I'm getting tired from how some people tend to do business in this country.

So, from now on, every single project I lead there will be no excessive tolerance to incompetence; here's how it is, from now on:

The three strike rule:

A person gets three and only three chances: the first is when he/she gets the job, then there are two possible misbehaviors (missed deadlines without a reason, recklessness, quality below ones level, ... etc.) and the person of off the project, period.

Wednesday, January 07, 2009

The evolution of the programmer...

A little story/joke I tell my students to explain the good and bad motivation they will face and to teach them where to look first:

There is a term in Russian programming culture called the "programmer laziness" (программистская лень) and it is a good thing, if one knows how to control it, that is.

In general, it is exhibited by spending hours to automate the solving of a problem that would otherwise take five minutes to fix by hand. When this problem arises again most programmers will again dig in for hours but, some more advanced people will remember that they've already did it, will blow the dust off their old code, adjust it and run it again spending less time than it took to code in the first place but, still substantially more than it would take to do the work by hand. In turn, most of these people will repeat the process again and again every time they face the task, evolving their code. But, a fraction will notice a routine and write a problem detector virtually eliminating the time spent on even considering the task (effectively, again, taking them to a new level of abstraction) only until a sufficiently different problem is encountered and then the process needs to be repeated from the start. And so on.

That was the first evolutionary level.

People who either have evolved to the second level or were born into it will say: "Oh! now, I'm seeing this too often, I wonder what is causing it?" as soon as they see an error for the second time at max.

Friday, November 14, 2008

a new tool! :) ...quite useful :)

it does this:

a very nice way to host and publish code!


Saturday, September 20, 2008

web and computers...

One interesting tendency about the web these days is dramatic browser tech advancement.... just a mad idea, could this lead to practical self-applicability and/or developing the whole system in (or rather as) a browser?

And I do not mean things like Dan Ingalls' Lively Kernel, no. The idea is more of a standalone bare browser/server on hardware that can be useful both connected or not. this could include a proxy cache as store; booting from the web or local server (loading data to the cache), working with the data locally and posting changes when the system is on-line.... redefining the personal computer concept in terms generally associated with the web.

Some food for thought :)

Sunday, September 07, 2008

OOP and framework cooperative reuse...

I've been hitting this problem for a couple of years now...

here it is, in short: we have an algorithm or a set of algorithms that operate on one or several "terms", this is usually called a framework and implemented as a relatively monolithic (usually but not necessarily) entity with several customization points, and is used by replacing/customizing these points to achieve a variation of the provided functionality. there are numerous examples of this approach among which are the Python's logging package and the Twisted framework, to name a couple of big ones, on the smaller scale there are things like UserDict, ...etc. this approach works quite well but has two rather big weaknesses:

  1. the bigger things get the more unnecessary complexity of design and implementation there are and in some (if not most) cases this also relates to complexity of use...

  2. a framework is extremely hard to to implement in a way for it to be reused several times within one context...

the second point may need a little explaining. let us consider this example: we have several frameworks that simplify pythons dict creation and use, these usually require the user to define only tree or four special methods and implement the rest of the dict interface on top of that... quite straight forward, indeed. so to demo the above concept let's try and implement an object that acts as a dict and at the same time responds to a special attribute interface, both protocols access different data stores and are essentially independent, but are almost identical in their base functionality.

so some of the traditional approaches to solve this are:

  1. implement everything by hand.

    as far as my experience goes this appears to be one of the most popular ways to solve things. this is quite boring and bad so I'll skip the details and code example.

  2. extend and wrap (the OOP way).

    In other words make the object a dict-like by extending a special framework and proxy the attr access to a nested container (or vice versa).

    here is an example:

    import pli.pattern.proxy.utils as putils
    import pli.pattern.mixin.mapping as mapping

    # mapping.Mapping is quite similar to UserDict form the standard library but is quite a bit more
    # flexible. though, for this example it makes almost no difference except for the base methods
    # needed by each library are a but different.
    class DictLikeWithAttrs(mapping.Mapping):
    _dict_data = None
    # to add some fun, all of the instances have the same "base" state, yet still can store private
    # data in their local namespace...
    _attr_data = {}

    def __init__(self):
    self._dict_data = {}

    # proxy the dict interface...
    ), '_dict_data')

    # proxy the attr methods to the dict interface of the _attr_data...
    ('__getattr__', '__getitem__'),
    ('__delattr__', '__delitem__'),
    ), '_attr_data')
    # and we need to take special care with __setattr__ to avoid infinite recursion...
    def __setattr__(self, name, val):
    if name in ('_attr_data', '_dict_data'):
    return super(DictLikeWithAttrs, self).__setattr__(name, val)
    self._attr_data[name] = val

    # this will set local attrs that have priority over the global state...
    ('setlocal', '__setitem__'), '__dict__')

    this approach has both it's advantages:
    • involves far less work than the first approach.

    • also is simpler, if proper tools and discipline are used.

    and disadvantages:
    • the result is not too reusable or configurable.

    • not simple enough (IMHO).

    • scales badly -- we essentially have to proxy all the methods by hand.

    • another reason why this scales badly is that you can extend only once per protocol and for each consecutive use you need to nest.

    just imagine how would an object that supports three or four different mapping-like interfaces look like...

  3. write an interface factory.

    let's go straight to an example:

    def mappingproxy(target, get='__getitem__', set='__setitem__', delete='__delitem__', iter='__iter__', depth=1):
    map = ()
    # these will enable us to disable some methods...
    if get != None:
    map += ((get, '__getitem__'),)
    if set != None:
    map += ((set, '__setitem__'),)
    if delete != None:
    map += ((delete, '__delitem__'),)
    if iter != None:
    map += ((iter, '__iter__'),)
    # sanity check...
    if map == ():
    raise TypeError, 'to generate a proxy we need at least one methods enabled.'
    # now generate the proxies...
    putils.proxymethods(map, target,
    # this sets the namespace depth of the caller... (an off-topic, but in case you wounder :) )

    def attrproxy(target, get='__getattr__', set='__setattr__', delete='__delattr__', iter=None, depth=1):
    return mappingproxy(target, get=get, set=set, delete=delete, iter=iter, depth=depth+1)

    so our class will look like this... (the changes are in italic)

    class DictLikeWithAttrs2(mapping.Mapping):
    _dict_data = None
    # to add some fun, all of the instances have the same "base" state, yet still can store private data...
    _attr_data = {}

    def __init__(self):
    self._dict_data = {}


    attrproxy('_attr_data', set=None)

    # and we need to take special care with __setattr__ to avoid infinite recursion...
    def __setattr__(self, name, val):
    if name in ('_attr_data', '_dict_data'):
    return super(DictLikeWithAttrs2, self).__setattr__(name, val)
    self._attr_data[name] = val

    # this will set local attrs that have priority over the global state...
    ('setlocal', '__setitem__'), '__dict__')

    this is a good approach, we made the user code both shorter and more understandable (be it at an expense of adding some more complexity to the library). but, on the down side, our factory and generated code are rather static.

    in essence, by automating part of the process we made the whole thing less flexible.

  4. design the interface "stackable" form the ground up.

    this is really bad as it makes everything dramatically more complex by adding dispatching to everything and confusing option and data registries/dicts/lists...

  5. macros or templates.

    I'd avoid this at this point as I'm investigating a possibility of a clean OOP approach.

I regularly use the second and third approaches but, I'm looking for something that would feel more natural...

In my view the best way to go would be defining a framework in a way that would isolate the algorithms from the terms they use, making the later overloadable.
On the small scale this is the same as a simple function, where the functionality is black-boxed, the user just inputs the arguments and gets the result; but this does not scale to collections of functions or frameworks with ease. I can't seem to find anything that would fit this in any of the systems/languages I know of (but I could be missing a thing or two).

Any Ideas? :)

P.S. you can find pli here...

Wednesday, August 27, 2008

the simplest design is allot more fun!

The first time I noticed this, I got quite surprised... the best solution for a given problem is the simplest, most straight forward, least thrilling and usually generates remarks like "this is textbook stuff, too simple and uninteresting" and, in fact, that it may well be!
but as a side-effect of simplicity one gets tiny code base and flexibility unrestrained by bells and whistles. The result is as complex as the task it solves without any extra automation and framework magic :)

Yes, you get less of that Perl-ish "I've written this but, even I do not understand how and why it works, but it does!" thrill and none of the satisfaction it brings...

but getting the job done in a fraction of the time and watching the client try to rip it apart and reject it as being too simple, too abstract or too optimistic and not being able to do so objectively, is allot more fun ;)